Influence of Nonlinearity on the Limit-cycles of Neural Networks with Asymmetrical Cyclic Connections

We have analysed the effect of nonlinearity of an activation function on the oscillation of the neural network with asymmetrical cyclic connections by using the averaging method. The oscillation obtained by linear approximation, in which the activation function is approximated by a linear function, is multi-modal although the actual oscillation with numerical simulation is unimodal. When the activation function tanh is approximated with a function that includes a nonlinear term, we show that multi-mode oscillations are unstable by using the averaging method.